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Today we are proud to deliver a special 
BSD Mag issue with compilation of all 
the best David Carliers articles. David 
has been cooperating with us for a cou- 
ple of years. During this time he has de- 
livered a great number of very technical 
articles, with thousands lines of code. 
His articles have always been in line with 
issue themes and we have a feeling that 
whatever topic we would ask for, he is 
able to write about it. He possesses 
great knowledge and passion to open 
source systems, mostly BSD - of course. 


We have chosen his best articles about 
FreeBSD, NetBSD, NodeJS, Hard- 
nedBSD, FreeBSD Security and Cloud. 


If you have been wondering who David 
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Security 


A secure webserver on FreeBSD with Hia- 
watha 68 


In most cases, when it comes to choosing a 
web server, Nginx comes quickly to mind (I 
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Using the FreeBSD's procstat API in a 
web context 


Among all the numerous specific features of FreeBSD, 
there is a famous command line to dump the statistics of 
the various current processes, procstat. Its internal API is 
fortunately exposed via the well named libprocstat library. 
Let's imagine we want to display it via a web page so for 
this article, we re going to use CppCms, one of the good 
quality C++ web development frameworks with a current 
FreeBSD 10.2 release version. 


1. Procstat API 
The list of the available functions can be viewed in this page 


https:/www.freebsd.org/cgi/man.cgi?query=libprocstat&sektion=3&apropos=0&manpath=FreeB 
SD%2010.0-RELEASE 


We just need to include the necessary headers and to link our application to the shared library 
libprocstat, simply. For our basic procstat service, we will expose the pids, the paths of the proc- 
esses and the owners of those. 


2. CopCms 


We could have used an usual full PHP solution, calling procstat utily via a system call, possibly 
parsing the output and displaying it. However, doing web development via low level langages is 
also possible especially in the embedded environments where the resources usage count. 


CppCms has a package, so pkg install copcms (or via the ports) is sufficient. This framework 
has a lot of useful features, session handling, caching, native encoding handling. For our basic 
usage, we'll use their advanced template system with the addition of jQuery to make it more ap- 
pealing. 
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Let's start with the template's content. For this purpose we need a C++ prototype and a CopCms 
template file. 
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ProcinfoContentSkin.tmpl: 


// For who has experienced various templates solution for Java, PHP and so on, some parts 
seem pretty familiar 


c++ #include "proclist.h" %> => We include simply our C++ prototype here 


<% skin ProcinfoContentSkin %> => Useful when the template are shared libraries 
<% view ProcinfoContent uses content::ProcinfoContent %> 
<o template render() %> 
<em> 
<head> 


<link rel="stylesheet" 
href="//jqueryui.com/jquery-wp-content/themes/jquery/css/base.css?v=1 
Ws 


<link rel="stylesheet" 


href="//jqueryui.com/jquery-wp-content/themes/jqueryui.com/style.css" 
> 


SSCeIPimsor@e— 1, COCs ue). com aGue ty le wlOn ee ie es Serine 


fee 1a alene 
Bie COME. ue my. Com, tay Wiehe A ice Uma ew) Ser ie 


<script type="text/javascript"> 
Sig Ce Gugiereubrenal (lean 
CA VEDOOy i sOmrao le (ju 
Si tbody” )) drsableselection ( 
b); 
Ga Oied nee; 


</head> 
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<body class="jquery-ul page-template-default"> 
<hl>Processes statistics</hl1> 
Colby, Jeulelars—Meleneueslatialsiey > 
<div id="content-wrapper"> 
<div id="content"> 
<table class="ui-sortable"> 
aii 
<i PE el 
<th PATH / tii 
<th>ARGUMENTS</th> 
<th>OWNER</th> 
bie 


——EDody— 


<% foreach info in pinfos %> => Iterate through the pinfos member of 
the content's class ... 


<tr> => ... then ‘echoing’ each field of a Procinfo struct 


<td class="ui-state-default ui-sortable-handle"><%= 
nei 97 cl = 


<td class="ui-state-default ui-sortable-handle"><%3%= 
.pathName %></td> 


<td class="ui-state-default ui-sortable-handle"><%= 
PeeCisie =< (cr 


<td class="ui-state-default ui-sortable-handle"><%= 
UserName. (<—.—sIntO,userEuliNameoe. \a—.—siInto.useriome:.>—7 car 
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<10stream> 


<sstream> 


oe. a= 


Sleeve ileal: 
<sys/param.h> 
<sys/queue.h> 
<sys/socket.h> 
Rae scrila a 


<sys/types.h> 


<sys/user.h> 


#include <pwd.h> 


#include <libprocstat.h> 


#include "proclist.h" 


eill cision JeNgele(sieche G2 oblollanise [sje oleniis arel 0) ollimberheukounl 7 
private: 

DROCS Cae ps: 

GOnrenis SProGintoconcenk spc; 
jejetedialen 


PrOoGcsvan(CODeMs = SeOrvVice cory) S .eopehns app llGarLon (ery) 
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// We're opening the processes info via the internal sysctl system 
// There are other ways, via a kernel's core dump file or via kvm ... 
Psts DLOCSsral veopenesysce Ew 

} 
Ase (OKeren Esmee) | 
PrOceclarecl ese (Ps)e, 


} 


Waliceiblek  omkel auitcbiensurels qisveredigley abuclL) ia 


MIS CON cnet Keleigncie: Aveo acl fee iercre aie cl “enh a 


Const kKintogproc + ka ow Mee pine lca sit COMsiig iMisom iOS 


eal er 


Cons Ge kriibe Romer a. ki} ide MMINUEIeNa| Ig Ne Verse “(logis Go: ligiblon jon aeve 
*>(b); 


Di (kea= > krgetds< Ko] kin pid) 


rebirn —— ls 


igiedzabhegie ike 
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void 
PROC sieae + Matic mel sc emda 


UNS Toned ants eu; 


aii@iiG sake 


// we just get the processes information w/o their thread IDS though ... 


/! We could get also only a specific group of processes per TTY or user etc ... 


aI@URe) JeNaelen = <joy —— V oldorersyercne, (efsiejouaelersy|(ersi 
SHOE) ee 


KERN PROC PROC, 0, 
fie, (Ne) =e ON a thay 


Igeueiehel ge 


De.pInlLos. — 


SLC 2VeCrtorcorrocimiro-> (ji 


qsort(kp, ct, sizeof(*kp), kp compare); //As the processes list is not 
ordered, we do per PID 


aS ele poe encs) a 
ehar path [PATH MAX]; 


PEOCsEataGeepaciname (pe, ck Lily path, ws lzeor maul), 


iis elem (oa) ee elt 
PeOGAa romps, 
aes cle eo: [at kaa ourel 
(Sue sfereneioull elite “wen Mass ene sige (erenelally, 
SEds "SErIngstream ss; 


// Here we get the possible arguments the process were called with ... 
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// args NULL terminated list pointer will be freed by procstat_close later 


Slayahign “Pavseeish Sh) Sigeles sue Wopeubrenciesw Woks a. Sle. Pal oF 


Gigsiier woo ialers, —— eliaie rst 


// pargs[0] == path here, so it is bypassed (hence we could have just used 
procstat_getargv ...) 


while (*++pargs) 


Si Oe eed o. 


1G AlroWGlG fof FSBO My aisipie imahe A repoustouece 4())e)) a 
passwd pw, *res; 
memset (&pw, 0, sizeof (pw)); 
Gigthie Aeyrhe MOVs e |e 
// Just to get more “human readable” process' user info 


ands VAVeloue one mkok jad C qerliala| alee ve biiyel ea as jonne oie: 
sizeof (buf), &res) == 0) { 


pi.userName = 
Sede seeing (pwaow name): 


pi.userFullName = 
SiGe owl NegGew. PWacecoe):; 


jCubaiblels velo (Oiilee =i eiwiels sesvenesl or jeniajonte elaine 


PC spIMmroOs Cush. back (oin)s, 


} 


CppCms uses the popular JSON format for the configuration's file as follow for our example. 
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The possibilities of configuration are pretty rich, here we're using the internal web server but in 
production, it might be preferable to configure in FastCGI mode and allowing a guenine web 
server, like Nginx, handling the client's connections. 
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If we planned to compile the template as a shared library, we would need also to declare it in our 
config. For more precise information, please read this page. 


First, we need to “compile” the template file into a C++ code via a CppCms utilily. 


Then compiling altogether our CopCms' application with this template. Indeed, for the sake of the 
simplicity and as we have only one template, we compile it statically. 


| would advise to use at least a Makefile ... The booster's library is necessary for the template's 
system otherwise it is also possible to render a HIML content directly in the application's level via 
an usual C++ stream like here. 


Once compiled, we can finally launch our CppCms's application. 
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./cppems procstat -c config.json 


illustration SEQ "Illustration" \*Arabic 1: Here our sortable list of processes 


This is it, we can now read the processes list and rearrange the order in a fancy manner. There is 
a lot of room for improvements, hopefully, that might give some ideas to you, readers. | hope at 
least, that will give you also the curiosity to dig in more in the procstat's API. 
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If you usually program in Linux and you are considering a 
potential switch to FreeBSD, this article will give you an 
overview of the possibilities. 


FreeBSD comes with either applications from binary packages or compiled from sources (ports). 
They are arranged by software types (programming languages mainly in lang (or java specifically 
for Java), libraries in devel, web servers in www...) and the main tool for modern FreeBSD ver- 
sions is pkg, similar to Debian apt tools suite. Hence, most of the time, if you are looking for a spe- 
cific application/library, simply: 


without necessarily knowing the fully qualified name of the package, it is somehow sufficient. 


For example: 


pkg search php5 will display phpd itself and the modules, furthermore php56 specific version 
and so on. 


The main difference is, you are not forced to either choose the binary or the port but can have 
both if it suits your need, but keep in mind that compiling from source can take a certain amount 
of time to achieve, if that is an important point for you. If the ports tree is not already present on 
your server, portsnap fetch extract will fetch the ports tree for you by default in /usr/ports. Then, 
related to the software type described above, you just need to go to the related folder. For exam- 
ple, for installing phpd5 : 
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The second command, depending on which options you are going to choose, will display all the 
options available for each dependency (for example, if gd support is enabled, furthermore the op- 
tions for graphics/gd library will appear). 


However, most of the time, the binary packages are sufficient to cover most of the needs. 
2. Web development 


This is basically the easiest area to migrate to. Most Web languages do not use specific platform 
features, so most of the time, your existing projects might just be “drop-in” use cases. 


If your language of choice is PHP, luckily, this scripting language is workable in various operating 
systems, most of the Unixes and Windows. In the case of FreeBSD, you have even more different 
ports or binary package versions (5.4 to 5.6). In this particular case, you might need some spe- 
cific PHP modules enabled, luckily, they are available uatomatically or if the port is the way you 
chose, it is via the www/php5-extensions one. 


Terminal 
File Edit View Search Terminal Help 


php5-extensions-1.7 


bc style precision math functions F 
bzip2 library support 


CALENDAR 
CTYPE 
CURL 

DBA 

DOM 

EXIF 
FILEINFO 
FILTER 
FTP 

GD 
GETTEXT 
GMP 

+(+) 


Figure 1. PHP port and modules. 


calendar conversion support 
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input filter support 
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gettext library support 
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Of course, developing with Apache (both 2.2 and 2.4 series are available, respectively www/ 
apache22 and www/apache24 packages) or even better with Nginx (the last stable or the last de- 
velopment versions could be used, respectively www/nginx and www/nginx-devel packages) via 
php-fpm is possible. 


Outside of PHP, the same applies to Python / Django (www/py-django) and Ruby on Rails (www/ 
rubygen-rails), Python 2.7 and 3.5 (lang/python<version>) are available as Ruby until 2.2 (lang/ 
ruby<version>). 


In terms of databases, we have the regular RDBMS like MySQL and PostgreSQL (client and 
server are distinct packages: databases/ (mysql/portgresql)<version>-client and 
databases/ (mysql/postgresgql)<version>-server) and the more modern concept of 
NoSQL with CouchDB for example (databases/couchdb), MongoDB (databases/ 
mogodb), Cassandra (databases/cassandra) to name a few. 


Also, if you need to perform efficient Map / Reduce for Big Data work, you have either the well 
known Apache Hadoop or Apache Spark (respectively devel/hadoop and devel/spark). Last, if you 
ever need a search engine, Apache Solr/Lucene (textproc/apache-(solr/lucene)), Xapian 
(databases/xapian) and their various language bindings are available. 


PhpProject1 - NetBeans IDE 8.0.2 


File Edit View Navigate Source Refactor Run Debug Profile Team Tools Window Help Q 


baal Fl 3) & Ee <default> x) e- i > A BR 


X|Files _| Ser. El /StartPage ~| [aj index.php x 4) >\[~) (0) 
% ee seca Source | History| MBM -B- QQGHBGG 2% 22 eo 2us B 
fa index.php g <?php ss 
© G@ Include Path 2 a 
3] class BlogPost extends BaseBlogPost 
40{ 
5 public function __toString() 
6 php PHP: Something Use’ x \ 4 
iu return $this->getTitle(); ite 9 
3 y S C65 php.net/manual/en/tutorial.useful.php 
10 
Q ?> 


Downloads Documentation | GetInvolved Help 


$HTTP * VARS arrays Instead, suchas $HTTP SERVER VARS . Although deprecated, these 
older variables still exist. (See also the note on old code.) 


To display this variable, you can simply do: 


Example #1 Printing a variable (Array element) 


<?php 
echo $_SERVER[ 'HTTP_USER_AGENT']; 
?> 


Figure 2. PHP development under Netbeans. 
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Is it rather Java Web or any language based on the Java VM platform? In FreeBSD, you even 
have Java 8 (either java/openjdk8 and java/linux-oracle-jdk18), various popular frameworks and 
J2EE servers or servlet engines like Spring (java/springframework), Jboss (java/ 
jboss<version>), Tomcat (www/tomcat<version>), Jetty (www/jetty)... Even the more 
modern languages like Scala (lang/scala), Groovy (lang/groovy) can be found. 


Two languages described above, Python and Ruby, have their Java VM counterparts, Jython 
(lang/jython) and Jruby (lang/jruby), available as well, 


In term of Integrated Development Environment, there are still several choices. The venerable 
Netbeans (java/netbeans or java/netbeans-devel), Eclipse (java/eclipse ... side 
note, FreeBSD needs to have Kerberos support enabled, NO KERBEROS is /etc/make.conf 
or /etc/src.conf presence needs to be checked) with their numerous popular plugins. 


3. Low level development 


The BSD is shipped with a C and C++ compilers in base. In the case of FreeBSD 10.2, it is clang 
3.4.1 (in x86 architectures), otherwise modern versions of gcc, for developing with C++11, for ex- 
ample, are of course available too (lang/gcc<version> ... until gcc 5.2). 


Numerous libraries for various topics are also present, web services SOAP with gsoap through 
User Interfaces with GTK (x11-toolkits/gtk<version>), QT4 or QT 5 (devel/ 
qt<version>), malware libraries with Yara (security/yara) ... 


In term of IDEs, Eclipse and Netbeans described above allow both C/C++ development, Anjuta 
and Qtcreator are also available for important projects. If you prefer, FreeBSD has in base vi and 
Vi Improved can be found in ports / packages (editors/vim or editors/vim-lite without X11 support). 
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Java - Eclipse SDK 
File Edit Navigate Search Project Run Window Help 
s @Welcome ® . ce Be 6 


2 + & 
C) Tutorials Samples What's New Migrate 


Overview 


Select a wizard — 


The Eclipse software development kit is the development envirol 


Create a new C project 


€ C/C++ Development 
a . 
Get familiar with the C/C++ Development Tools (CDT) Wizards: 
| 
type filter text 4 
— Develop PHP applications ee 
Se 
= Leam how to develop PHP applications by reading the PHP D: General 
ve C/C++ 
“a= Workbench basics aC Project 
[ Learn about basic Eclipse workbench concepts C++ Project 
@ Class 


ce) Team support 
Find out how to collaborate with other developers 


iS 
oD 
= 
oO 
x 
“a 
Vv 


Cancel Finish 


Figure 3. Java Eclipse SDK. 


FreeBSD is a POSIX system, hence porting C/C++ code to this platform depends on the degree 
of portability of your projects, so the usage of specific “linuxisms” and such. 


In case more information is needed about porting software in FreeBSD and its specific tools, | 
would recommend the reading of BSDMag issues no 66 and 68. 


4. Android / Mobile development. 


In order to be able to do Android development, to a certain degree, the Linux compatibility layer 
(aka linuxulator) needs to be enabled. Also x11-toolkits/swt and linux-f10-gtk2 
port/package need to be installed (note that libswt-gtk-3550.so and 
libswt-pi-gtk-3550.so are needed, the current package is versioned as 3557, can be solved 
with symlinks). In the worst case, remember that bhyve (or Virtualbox) are available and can run 
any Linux distribution smoothly. 


21 


MAGAZINE 


BSD 


The FreeBSD Corner 


Packages Tools 


SDK Path: 

Packages 

w Name API Rev Status 

y¥(i Tools Pot 

w¥# Android SDK Tools | |4.4.\@ Installed 

¢# Android SDK Platform-toa '3.0..5 Not installed 
+ Android SDK Build-tools | 3.0.) Not installed 
# Android SDK Build-tools ___'3.0.. = Not installed 


Android SDK Manager Log 
STON nn rorennin gy -Grenin prnca ror ahaa re oor 
Installing Samples for SDK API 23, revision 2 
Installed Samples for SDK API 23, revision 2 
Downloading Sources for Android SDK, API 23, revision 1 
Installing Sources for Android SDK, API 23, revision 1 
Installed Sources for Android SDK, API 23, revision 1 
Downloading Android TV ARM EABI v7a System Image, Android API 23, revision 2 
emeee Android TV ARM EABI v7a System Image, Android API 23, revision 2 


Installing Android TV ARM EABI v7a System Image, Android API 23, revision 2 Close 


>» Om Android 4.4.2 (API 19) 
> fAndroid 4.3.1 (API 18) 
» Of Android 4.2.2 (API 17) 
» om Android 4.1.2 (API 16) 
> f2Android 4.0.3 (API 15) 
> Of Android 2.3.3 (API 10) 


» of Android 2.2 (API 8) 


Show: &% Updates/New “Installed Select New or Updates 
Obsolete Deselect All 


rw ST we = rar rawr 


Installing Android TV ARM EABI v7a System Image, Android API 23, revision 2 oO? 


Figure 4. SDK Manager under FreeBSD. 


5. Source Control Management. 


FreeBSD comes in base with a version of subversion, as FreeBSD source is in a subversion re- 
pository, prefixed svnlite, though, to avoid conflicts with the package/port. 


In addition, Git is present but via the package/port system with various options (with or without a 
user interface, subversion support). 


6. Conclusion 


FreeBSD has made tremendous improvements over the years to fill the gap with Linux whereas it 
still Keeps its own interesting specificities, hence there would not be too many blockers if your pro- 
jects are reasonably sized to consider a migration to FreeBSD. 


22 


MAGAZINE 


BSD 


Moving from Linux to FreeBSD involves quite a number of 
changes; some gains and some losses. As a developer, for 
most of the programming languages, especially the high 
level ones, there are no meaningful disturbing changes. But 
for languages like C (and its sibling C++), if you want to 
port your software, libraries, etc., some points might need 
to be considered. 


- How to move from Linux to FreeBSD - Basic knowledge of C programming 


- How to develop under FreeBSD 


As is often the case with C, it is not especially straightforward; the code itself might need some 
changes, minus the pure POSIX part. Let’s say your program needs to use some known network 
functions. 
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#endif 
#include <sys/types.h> 


#include <sys/socket.h> 


#include <arpa/inet.h> 


int 
(ee tbiay( Galaga “vetiae (owe seleveiia  urciiaeny|| J) 


{ 


S Gene es iehiaclG tee leh; 


COMstaeiem 1 ea kon 


Hise ONG Siw ouunene! (VUsa EINE ike asl gh) ey ail) 


jairekiliald CikigNe (elie 6 (One @lalsma sromaele pve ly) 


{ 


SErUC baht roG mint. 


char hwaddr [6] 


In addition, FreeBSD provides a bunch of specific functions, like stricpy/stricat (safer versions of 
strcpy/strcat) and strtonum family functions, all of which are available in the base, whereas Linux 
must install the separate BSD library to have them. If you have any doubts about any functions, 
all manpages are available and very well written. 


FreeBSD is shipped by default with clang, whereas Linux relies on GCC suite. If you heavily use 
OpenMP, clang does not provide it yet so you might need to install GCC from ports. Somehow, 
clang mostly compiles faster and provides more informative warning and error messages. Fortu- 
nately, they share a significant amount of common flags. 


On Linux, you may use a custom memory allocator during your development, like jemalloc. It’s a 
very handy and useful library that allows you to generate statistics, to fill freed memory with spe- 
cific values, and to spot corrupted memory usage. 


Good news! You do not need to install it—FreeBSD libc’s malloc (aka phkmalloc) uses jemalloc 
internally. To print statistics from your application, for example, you need to include 
malloc np.h instead of jemalloc/jemalloc.h 
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As for the makefiles, this is the BSD format which differs from GNU style: 


A basic makefile for a library: 


A basic makefile for an application: 


BSD 


MAGAZINE 


26 


FreeBSD can handle GNU via (gnu)make, libtool, etc. via the ports. 


Or to save the effort of porting this part, it might be more handy to use cmake or scons. 


You might want to publish your library / application in pure FreeBSD’s path. You can make a port 
that can provide some options for the user. It can download the source and compile it with its de- 
pendencies in a natural manner. In addition, you can build a binary package to facilitate the distri- 
bution. 


john.doe@email.com 
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For instance, you can put the archive .tar.gz of the library in /usr/ports/distfiles, then 
type make checksum. Then, make install will compile and install it in /usr/local. The handbook 


of making ports is very useful to read. 


Furthermore, you can build a binary version of this port to facilitate its distribution. As simple as it 
is, pkg create myli. It will create a txz archive in the current folder. In the end, pkg _ install mylib 


will install it. 


Developing under FreeBSD is not the extreme challenge you might think it is. Even better, from 
coding to publishing, everything is thought out and made in a constant way without any external 
dependencies. If you want to go even further, like kernel development, again it is easy and in 
base. So there is no real reason to stay away from FreeBSD anymore, you are more than wel- 


come. 


The HardenedBSD project was created in 2014 by Oliver Pinter and Shawn Webb. The project 
aims to provide security enhancements to the FreeBSD project. We plan to upstream most, if not 


all, of our projects. 

The core HardenedBSD team consists of: 
¢ Oliver Pinter 

¢ Shawn Webb 

The developer team consists of: 

¢ David Carlier 
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e Nathan Dautenhahn 
¢ Danilo Egea Gondolfo 
¢ Oliver Pinter 


“Shawn Webb 


The following people and organizations have contributed to the HardenedBSD project: 
¢ Ilya Bakulin 

¢ Bryan Drewery 

¢ Danilo Egea Gondolfo 

¢ Dag-Erling Sm@rgrav 

¢ Robert Watson 

¢ Hunger 

¢ SoldierX - Donated a sparc64 and a BeagleBone Black 

¢ Hyper6 - Designed logo 


¢ Automated Tendencies - Substantial monetary donation. 
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In this article, we will have an overview of what is called a 
syscall (system call shortened), from the kernel side to the 
userland then in the end how to create a new one, in 
FreeBSD. 


It is assumed you know how to build FreeBSD-current and 
have some knowledge about C language. 


A syscall is a code implemented in the kernel side that can be potentially called from the userland 
for various purposes, network, generating some random data or controlling the system processes 
as well. Whatever the type of syscall, the userland never interacts directly with the kernel but via a 
defined interface. Let's start with a simple example, the getpagesize call, which is the number 
of bytes per page. 


Let's see how, in the kernel, it is implemented. 


Which simply returns the constant via the sysctl system. Let's see now how it is implemented in 
the userland side. 
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lib/libc/gen/getpagesize.c 


nnahe 

getpagesize (void) 

{ 
sige. aiah only |p 
static int value; 
SUVA ye Gstilwaerr 


Litt Siow. 


if (value != QO) 


return (value); 


/* Here we try to get as much of the cached result as possible */ 


sieiiene 2 Mem choc aligune) (Gadhia Clean ke rccuible) sealvasiess (vecuLil=s) 1 
if (error == 0 && value != 0) 
return (value) ; 
/* Otherwise we get the result via the sysctl call */ 
mib[0] CTL HW; 
mee [5] HW PAGESIZE,; 
size = sizeof value; 
PP MSVYSscel (mbes, «value, «size, NULL, 


ie Siete rie (el) as 


As you can see in this case, mainly the work is done in the userland side. Another type of syscall 
exists where all the implementation is done in the kernel side only but its symbol is transmitted via 
a symbol map. Next, let's have an eye on the getpid syscall. The usual C function has this 
signature. 


Those syscalls are organized per type in the kernel, implemented in the files 
src/sys/kern/(kern/sys)_ *.c. Basically, the sys_* files hold the function implementations 
but if the complexity requires it, some codes are split in the kern_ * ones. 


First of all, the struct getpid_args reflects the userland parameters, every syscall needs a cor- 
responding *args struct to pass the userland parameters, hence as getpid has none. Here 
we have a dummy's. 


This type of syscall has this signature: 
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The thread parameter is the current thread from which we can get the current process, locking it 
in order to modify or just read a specific state. The integer returns here to, potentially, set errno in 
errors code paths. 


Here’s the corresponding getpid implementation. 


In the case of getpid, it is always successful, errno is never set so we return 0 directly. 
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PROC TEOCK (a); 
tabs Gel eh ellie ton de MSV NG laID A 1) * 

Dp tEa = Ps DeOP ER. Dupic, 
PROC UNTOCK (po): 

} else { 
PROC UNLOCK); 
SxS LOCK (Se EOCureeR Lock) y 
Pipa OPOCr Gear p arene 
ppid = pp->p_pid; 


ob. d PetbheLoie<(Usendereiescice Ak@el.<)) 4: 


issue Hela | Gejosiel) 2 


/* Returns the real parent process whether it had exited or 


traverses the orphaned linked list */ 


Sie te Cee Can 


Creole! ek Scull eet (sie ribisie Aoleere! = elaal kel) 


{ 


SETMCEMDEOC wo, as Oar eile: 
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SPs sere Spr Oerree el OC AP LOChED):, 
aie) AWelaab ele er eres leete, Gores. J ledesh (Oladelait ley)» pas 10))) si) 
abn Mile qabdbelae ch 36) 0) okie he | | 
Chik ORO GS One ide == Seniske popped) 


parent (lab lkio Gee), 8) 95) ay 


parent dima @ er. 


return (parent); 


(p Ciel (Oa Ombseer Lage ee TRE Rh tO PHAN 0; aA 


/* Cannot use LIST_PREV(), since the list head is not Known. */ 
Dea eCOMceINeTOL (S— peorolal, bos rew mse Cup LOC, 
PLObpnans tesnext); 
KASSERT ((p->p_treeflag & P TREE ORPHANED) != 0, 
(Hissin PeOR PHAN sO 07 iO) ja, 
} 
Sieleiene een sigaicncone (We) Pes onc elelsighs lee fergcnign Gs enculors "oneerey 
(ob Kimo] gvehavel allot se alicioue } ey 


return (parent); 


Now, the next step to see is how the kernel makes sys_getpid available to the userland. 


Our getpid syscall can be found in sys/kern/syscalls.master file 


20 is the syscall identifier, AUE_GETPID is the auditing event. Auditing in FreeBSD is the possibil- 
ity to log different type of informations from those syscalls: authentication, file descriptor opera- 
tions (via fentl call), userid/groupid changes. In our case, in fact, AUE GETPID is equal to 
AUE NULL, which means no auditing. At last, STD means it is a standard syscall. 


FreeBSD allows you to add new syscalls similarly to the ones above. 


For the example, let's make a syscall that returns the full command of a given pid (and if the pid is 
-1 then it would be the current one), which would have this signature: 


It might be advised to create a new file instead of updating sys generic.c, let's create 
sys _custom.c inside sys/kern folder 
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#include <sys/sysctl.h> 


#include <sys/sysproto.h> 


// Here this is just for the example, having a “optin” to enable our custom function 


SWeneMies aide evmbstolls. vets qi) =? (0, 
SloCTRVTINitsdebug,] OlD AUTO,» enable custom, CrErhaAcraw, 


GCenavlevevst On, Ue ema vkereie Om): 


am gue 
eyo ecUSEOMMr UNG (SERUM CERENGead Ey eoerUCte cCUSrtOMn PMC wats stem) 
{ 

Sore le OmOCGe sor 


ihn iO ee — aes 


jOOl = wel aeere ves Ih ON 16h. 


memset (uap->buf, 0, uap->buflen); 


alae <A (aiake' oon We) ursyeroi 0 a 
// We set errno if the buffer is too small or 
if (uap->buflen < MAXCOMLEN) { 
OO) nee) eenenige ON yee 
error = EINVAL; 


GOLOmeliEs. 
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// if we do not find our specific process 
Ali GSK sy Se Owe aby pee 
pfind(uap->pid)) == NULL) 
eG lePaerol ge Sewell WN ears 
error = ESRCH; 
GOEOL. OWE, 
} 


PROC UNLOCK (p); 


td=7Edt proc; 


/! Before copying to the buffer the command, we lock the process 


PROC ELOCK (o)- 
Stricpy (uap- but, 9o-4 > sconm,. MA ~COMERN):; 


PROC UNLOCK (p) ; 


rekiurn (Error )r 


To declare our new function, we need to add this in sys/kern/syscalls.master file: 
Note: the identifier must be unique, we just increment from the previous. 
Then, in our FreeBSD source folder, we could type this: 


Now there are new entries in sys/sys/sysproto.h header, first we have our userland struct 
arg: 


and our kernel function declared: 


We need to declare our new file sys_custom.c to the sys/conf/files file as below: 
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So the next time we compile the kernel it will be taken in account. Last, we need to make avail- 
able this function exporting its symbols: 


Once we have updated our system, our new function and our new sysct1 ought to be available. 


We could test this new function with this short sample code. 
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If you think a new syscall is needed then you can either discuss it in the dev freebsd mailing list or 
proposing a patch. Maintaining locally this kind of code change might bring difficulty when updat- 
ing the source with new official syscalls being added and conflicting identifiers. At least, hopefully 
this article gave you the taste to dig into this topic. 
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FreeBSD Kernel 


In this article, we will give an overview of the nature of the 
FreeBSD’s kernel. The important configuration files will be 
explained in addition to learning how to compile the whole 
system with more options, with more debugging informa- 
tion. Very useful for kernel development. 


What do you need: 


« FreeBSD 10.x. 


- Machine with at least 4 cores is recommended for the system compilation. 


- Genuine hardware or virtualized environment as your convenience. 


1. The FreeBSD kernel 


FreeBSD, like many kernels, is a monolithic kernel with loadable module support. Hence, it is pos- 
sible to build FreeBSD’s kernel with all needed modules statically or, those ones that support it, 
as separated dynamic loadable modules. 


The latter ones can be loaded and unloaded at will at boot time (<name of kernel mod- 
ule> load=“YES” in /boot/loader.conf file) or via kldload/kldunload. 


To have an overview of all currently loaded modules, you can type kldstat. 


For example, the output looks like the following: 


Id Refs Address Size Name 


IO VOXPEEE EE EROOZO00COMPOL ES Oo ekernet 
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If we add dtraceall_ load=“YES”, we will be able to use Dtrace framework facility. You can 
find an excellent introduction of dtrace in the BSDmag issue of December 2014: 


Indeed, Dtrace can be very useful for tracing syscalls. 


In order to build the system, we need the whole source code, hence the kernel and the userland. 
The userland is simply all the base utilities of FreeBSD. Both kernel and userland code are consis- 
tent and tied together, available in the same subversion repository. Apart from pure BSD codes, 
we can find GNU libraries and software (called contrib code). In addition, for ZFS, CTF (Compact 
C Type Format debug section, similar to DWARF format but reduced in term of size) and DTrace 
proper compilations, some CDDL codes are present. Happily, FreeBSD is provided with subver- 
sion in base, suffixed distinctly to avoid colliding with the port version. 


Check out the source in /usr/src via svnlite 


svnlite co ht | /usr/src (or you can check out the 
current branch with much newer sone but with more instability, you can just replace stable/10 by 
hed). 
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SFreeBSDS 
On the handling of kernel options 


All kernel options should be listed in NOTES, with suitable 
descriptions. Negative options (options that make some code not 


compile) should be commented out; LINT (generated from NOTES) 
should 


+ COMpeLe tas Much nGCOUe asmpOsstole. Try to stoucture OplL von using 
code so that a single option only switch code on, or only switch 
code off, to make it possible to have a full compile-test. If 
NeCSSssathy, a VOuUscam checks tore COMPILING MINE sro ter maxamun code 


COW SICEIVO'S - 


All new options shall also be listed in either "conf/options" or 


Meontr/ Ope Ons..Machine > . Oot OnsmEidtrat rect as scinche ssOurce—t mire 


Kn he oes teulGrocecinecCrecin I EOml Opera ym WidicmOOt lon. 


EO CEE, webeime Crem he Men olor sedi ce Volcve ich Mouhelioim cle ae. “ojoe, ecole baal alae 
this 1s a kernel-wide option (used just about everywhere), or in 
LOOtE Corel atale = > bOWwe Be eace 7) weil Pear eCUomel hy sscMema des. 


Note that the effect of listing only an option without a 
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header-file-name in conf/options (and cousins) is that the last 


convention is followed. 


This handling scheme is not yet fully implemented. 


EOERMeEtE Od cehas et 1 le: 


Option name filename 


If filename is missing, the default is 
ODE “Nalle ef SOP ENON adn Lower > eace =i) 

INE DisisiUE: Wererw a reieie ia) 

AOR Ee DEBUG TO pimeeac Gacicds ia 

AHC ALLOW MEMIO opt_aic7xxx.h 

AHC TMODE ENABLE opt aic7xxx.h 

AHC DUMP EEPROM opt aic7xxx.h 

RHC VE BUGT Ol ealey xxx 

DHCeVEBUCTOR TS cop taale( xxx, 1 


AHC_ REG PRETTY PRINT opt aic7xxx.h 


ADE DEBUCG Ope yedG 7) Ixx i 


ADVE BUG TORTS Compate?] 2x 
AHD TMODE ENABLE opt _aic79xx.h 


AHD REG PRETTY PRINT opt aic79xx.h 


¢ For each line, the option’s name then the file created with the relevant preprocessor defined. If 
the option is present in your kernel configuration file, let’s say AHD_DEBUG OPTS, it is possi- 
ble to test if AHD_DEBUG OPTS is defined and providing some contextual code for this option. 


¢ Let’s imagine we did a new shiny kernel module, we could add our proper line in this file. 
BSDMAG opt bsdmag.h 


¢ Another important file is /usr/src/sys/conf/file. 
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¢ For each line the relative path to sys, the type of module, if optional it will be compiled with the 
(lower case) option name written afterwards. 


¢ Again, with our new module, we can add in this file our specific kernel module C file, let's say 
workshop modulel. 


So now we can create a custom kernel config. Let’s call it WORKSHOP. 


(it will pick up the new WORKSHOP configuration file, by default it is the GENERIC one) 
Steps to build a system: 


First, the userland needs to be compiled. 


If your machine has multiple cores, it is advised to use them for the system compilation. 


It might take several hours depending on your current configuration. 


Possibility to do make -j<number of cores+1> buildworld kernel (ie create and install 
the kernel at once). Restart in single user: 
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Eventually mergemaster -p then make installworld: 


mergemaster -FUi => mergemaster will try to merge various configurations files and asking 
you how you wish to proceed, merging as possible, replacing with a newer one or keeping the ex- 
isting. 


Restart in normal mode. 


You should have now a workable system with the latest fixes/patches for the 10.x branch. But, as 
a developer, we might need more info from the system for debugging, studying the core dump af- 
ter a system crash/kernel panic. It is advised, as kernel developer, to enable kernel core dump 
writing (could be enabled when you installed FreeBSD or, afterwards, can be enabled via dumpdir 
rc.conf variable) at the cost of disk space consuming (can be potentially important, deleting old 
ones is necessart). They are, by default, located in /var/crash. In order to debug a kernel crash 
dump, the kernel compiled with debugging symbols, kernel.debug, is necessary. gdb can already 
be used this way. 
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Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. 
Loaded symbols for /boot/kernel/dtraceall.ko.symbols 

Reading symbols from /boot/kernel/opensolaris.ko.symbols...done. 
Loaded symbols for /boot/kernel/opensolaris.ko.symbols 
Reading symbols from /boot/kernel/dtrace.ko.symbols...done. 
Loaded symbols for /boot/kernel/dtrace.ko.symbols 

Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. 
Loaded symbols for /boot/kernel/dtmalloc.ko.symbols 

Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done. 
Loaded symbols for /boot/kernel/dtnfscl.ko.symbols 

Reading symbols from /boot/kernel/fbt.ko.symbols...done. 
Loaded symbols for /boot/kernel/fbt.ko.symbols 

Reading symbols from /boot/kernel/fasttrap.ko.symbols...done. 
Loaded symbols for /boot/kernel/fasttrap.ko.symbols 


Reading symbols from /boot/kernel/lockstat.ko.symbols...done. 


Loaded symbols for /boot/kernel/lockstat.ko.symbols 


Reading symbols from /boot/kernel/sdt.ko.symbols...done. 
Loaded symbols for /boot/kernel/sdt.ko.symbols 

Reading symbols from /boot/kernel/systrace.ko.symbols...done. 
Loaded symbols for /boot/kernel/systrace.ko.symbols 


Reading symbols from 


/boot/kernel/systrace freebsd32.ko.symbols...done. 


Loaded symbols for /boot/kernel/systrace freebsd32.ko.symbols 


Reading symbols from /boot/kernel/profile.ko.symbols...done. 


Like the userland gdb’ s counterpart, we can use backtrace (bt). 
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#o Ox0000000000000000 in ?? () 

Also list if we want to see 

(hodbigeitsime scr rit irr Wo Say 

OR EEEEEEPESO95oa4 7) Wiss ine Ss leepe(/Usr/src/sys/kern/ Kern rsymen.c2254)), 
249 else if (sbt != 0) 

20 ey lees Ce og et imedwalh (ident pl); 
ZS Velse rr SCCatcm) 

Z2o2 eval sHileis) Cle) Wellies, setae {(aGelei@aey. Geiad Ge 
293 else { 

oe eS hooey Wellhead Cle lr, eledaye; 

Vo Oe SEN iene 7 10 

Zoey 

257 #ifdef KTRACE 


Co oem ietem (Bin PON titel abn a, i) 


Then, for example, going up in the stack frames calls and so on... 


(Cieejele) blak 2 

Po OUXET ETE r ES UISoaD / Vine sleep. (adent—Ux)  lock—-Uxtietrou0ussecasu, 
DPraOority—O0, wmesg—OxftttfrrrrsOrr4ayt2 "—" sbt—0, pr—0, 

flags=<value optimized out>) at /usr/src/sys/kern/kern synch.c:254 
25a ee Leepo Walt(ident, El); 

Gsepoioy © dabeoe 

249 else if (sbt != 0) 


Hoi) ietecid. |e fedcievere/ Vie iyi eolelied h@lei@ueys Melieik CP 


For more information about gdb, a good helpful workshop exists about this topic: 


Indeed, especially if you run the -CURRENT branch, the kernel can crash for various reasons and 
this gdb like tool is handy to have a basic understanding of the reasons... 


Indeed, FreeBSD does not rely on the Giant Lock model anymore, it is based on fine grained 
level process locking/unlocking. Hence the resulting programming can be tricky and it is easy to 
get lock contentions. 


With this workshop module, you learned the basics of kernel custom configuration, compiling the 
whole system. 


kgdb 

GNU gdb 6.1.1 [FreeBSD] 

Copyright 2004 Free Software Foundation, Inc. 

GDB is free software, covered by the GNU General Public License, and you are 
welcome to change it and/or distribute copies of it under certain conditions. 


Type "show copying" to see the conditions. 
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There is absolutely no warranty for GDB. Type "show warranty" for details. 


This GDB was configured as "amd64-marcel-freebsa"... 
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NetBSD and pkgsrc-wip* 


For this mid-summer, we will approach a lighter subject, 
NetBSD and its ports system. Pkgsrc is the framework to 
build third party packages for this system. We will see how 
to create a package and hopefully submit it. Hence, the 
pkgsrc is supposedly already in your system. Otherwise, a 
full quide is available here. 


1. Environment 


It is recommended to install pkglint which will serve to produce a better package. Indeed, as its 
suffix suggests (lint, the historical C code analyser), it will check the whole package structure, the 
Makefile, the checksum and so on. 


Secondly, you need to choose a main category for your library or application, even if your future 
package can possibly recover several. For the article, we will choose security/yara, the popular 
malware searcher library, as an example. 


2. Makefile 
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WRKSRC= S${WRKDIR}/yara-${YAVER} => WRKDIR represents where the source port 
will be extracted (generally it is work/<package name>-<version>) 


USE TOOLS+t=pkg-config automake autoreconf => Necessary tools to build the 
package. Could be cmake, perl. They will be installed if not present 


USE LIBTOOL= yes 
GNU_CONFIGURE= yes => Uses GNU version of configure script 


PKGCONFIG OVERRIDE+= libyara/yara.pc.in 


pre-configure: 


cd S{WRKSRC} && autoreconf -fiv => Wecan override many sub tasks, related 
to different steps, before, after the archive extraction, configure, build, installation and so on 


-include "../../security/yara/Makefile.common" => Makefile.common is used 


by at least two packages (in our case py-yara) and it regroups common information, could be 
the dependencies, the version ... 


.include "../../mk/bsd.pkg.mk" => Mandatory file to include, it contains the main 
necessary variables 


Now, let's have a look at the Makefile.common 


# SNEtBSD: Makefile.common,v 1.3 2015/06/14 21:28:44 pettai Exp $ 
# 
# used by security/yara/Makefile 


# used by security/py-yara/Makefile 


We talked earlier about the DESCR file, it is simply a text file which describes more completely 
the package in question like below. 


We also need to know the list of files to be (un)installed relative to the variable PREFIX (usually 
/usr/pkg). It is the role of the PLIST file. 


57 


BSD 


MAGAZINE 


NetBSD 


bin/yara 

bin/ yarac 

include/yara.h 
inelude/vyara/ahnocorasick, h 
include/yara/arena.h 
include/yara/atoms.h 
include/yara/compiler.h 
include/yara/error.h 
include/yara/exec.h 
include/yara/filemap.h 


include/yara/hash.h 


include/yara/libyara.h 


include/yara/limits.h 
include/yara/modules.h 
mnielude/vyara/oojyect. nh 
include/yara/re.h 
include/yara/rules.h 
include/yara/scan.h 
include/yara/sizedstr.h 
ime hWwae my amea Ss ermine sl 
include/yara/types.h 
include/yara/utils.h 


ilabley Mlablenzenerceyd be! 


Sometimes, the software in question needs to be patched in order to work properly. The patches 
subfolder should contain the necessary diff files, by convention named patch-<path to the file, 
dashes replaces by underscores>. In our case, we have patch-libyara_ proc.c which just 
needs to add NetBSD support. The patchset is created via make patches. 
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Eventually, if it’s a library we can create the buildlink3.mk file, if another package needs yara 
library as a dependency, this package just needs to include this file: 


Once we have all the pieces needed, we can finally create our distinfo file which stores the check- 
sums of the DISTFILES and eventually the patches. It is created, ideally, via make makesum. 
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Size (yara/v3.3.0.tar.gz) = 7634474 bytes 


SN RVMIL “iUereleie!layslLaLlenccliggs’, ShaKoke aiey n= 
becO70ldega2 ] cesecd/S9GresaaldUZd0 las be 


7. Checking the package 


pkglint will display every part of the package which is not correct, the FATAL messages must 
be taken into account, some WARNING messages, too. 


pa ee platigye 


looks fine. => Ideal, but a correct package can have few harmless warnings too. 


8. Submit 


There is a project which aims to get more people involved in investing their time to create pack- 
ages for pkgsrc. It is called pkgsrc-wip and can be found here hito://okgsrc-wip.sourceforge.net 
and if your package is correct enough you can get commiter bit. | hope this article gave you the 
taste to create yours. 
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HardenedBSD, always ahead in security 


Previously, | focused on the HardenedBSD project, handled 
by Oliver Pinter and Shawn Webb, especially the Address 
Space Randomization Layout feature. The HardenedBSD, 
also has other features available and I'll try to describe all 
the features. 


1. arc4Random and Chacha 20 


Currently, FreeBSD uses the RC4 stream cipher for the arc4random family functions, both in ker- 
nel and userland side. These functions serve many purposes, for example, in the kernel side; it 
allows the creation of proper randomized processes id, the stack protection canaries, and the 
HardenedBSD Address Space Randomization Layout uses it as well. 


In the userland side; openssh uses it widely and is also used in the stack protection counterpart. 
It is generally an important piece of software. 


Recently in the last Hackfest (and previously in the last EuroBSDCon), Theo de Raadt discussed 
the arc4random OpenBSD’s version and raised the need to move on from RC4 to a stronger 
stream cipher. Hence, the invention of Chacha 20, implemented after the 5.5 release. 


Subsequently, we decided to update the HardenedBSD as well, in both kernel and userland side. 
In the kernel side, the challenge was to keep it SMP safe while keeping the code change smooth 
and wise while in the userland side, the challenge was to update the fork detection. Indeed when 
a fork is created, the reseeding is triggered. Usually, getpid function is used for this purpose but 
we thought there might be a better and more solid approach. M. Dempsey, an OpenBSD contribu- 
tor, provided a new minherit flag, MAP INHERIT ZERO to ensure that the memory map is prop- 
erly zeroed in this case. So, for HardenedBSD, anew INHERIT ZERO flag was added. 


Related to this, a new system called, getentropy was added as well. Basically, it fills a buffer of 
randomized bytes with maximum of 256 bytes. It serves more as an initial input for randomization 
rather than using it directly. Hence, for example, it can replace a couple of sysct1/KERN *RND 
Calls. 
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Again, we got inspired by OpenBSD and added some of their useful libc functions. 


getdtablecount gives the number of file descriptors per process. It can be helpful alongside 
getdtablesize. 
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HardnedBSD 


#define FDRESERVE 5 


ate 
Meda) (i Giteaa kOe, 6 Claires” aa Cn al)) 


{ 


if (getdtablesize() - getdtablecount() < FDRESERVE) 


Sree) VUnniinis OL On mi iel eee sc iu Os. an, 


reallocarray checks some potential overflows (but does not zerofy) 


ais aren ukolas e<= i areldliblenedqe: 


ie eles er i <i 


sane 
iivctltan Gigheuws defers ielars ys relays |p) 
{ 
stat eas (oe Chey 
// 1.€ same as realloc(NULL, 2 * sizeof(*p)); 


realteocarray (NULL, 2, SilZeor(*p)); 


A slightly different version of stricpy is provided. stricpy usually guarantees a zero at the end of 
the buffer. But the buffer does not sanitize the potential remaining bytes. So our version combines 
both stricpy and strncpy advantages at the cost of a slight performance hit, only HardenedBSD, 
at the moment, does it. 
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For the last, the crypt API was updated recently. Two new functions were added, 
crypt newhash and crypt _checkpass. The latter provides an easy interface to test the valid- 
ity of a password, while the first allows the creation of a hashed password. Once again, inspired 
by OpenBSD. 
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A secure webserver on FreeBSD with 
Hiawatha 


In most cases, when it comes to choosing a web server, 
Nginx comes quickly to mind (I personally appreciate this 
one a lot, no doubt about this). However, an interesting alter- 
native exists that embeds some very nice features, an alter- 
native called Hiawatha. 


1. Features 


What makes Hiawatha special? First, its code is well audited and famous for its solidness in term 
of security. Apart from having CGI/FastCGI support (hence possibly making dynamic website with 
PHP-fpm), SSL, lpv6, Virtual hosts. it also provides a protection against SQL injection and XSS, 
CSRF natively. It might lack third party modules support, as the architecture does not allow it but. 
Hiawatha has Reverse Proxy support! 


Luckily, FreeBSD already has a package / port. So once installed. 
2. Configuration 


Let's configure it. Here’s a sample configuration, a FastCGI's one. You can see that it appears 
very human readable. 


ErrorHandler AQAA = / 404) hen 


lewingholabare py | 


Port = 80 
Interface 


MaxKeepAlive 
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Security 


# Two first secure measures, limiting the client request size and the max time for 


# the maximum time for a client's connection kept opened 
MaxRequestsSize = 512 


TimeForRequest = 3,20 


Isubiave akiavey (| 
Port = 443 


Interface = 


SSLcertfile = hiawata.pem 
ReguireTLS = yes 
# Add X-Random header with a 256 value long 


RandomHeader = 256 


# A feature to make decisions based on url regexes 
Ohelkthoroue) <isiaes | 

ToolkitID = phprewrite 

Method GET 


Match /private DenyAccess 


Match “/page/(.*) Rewrite /index.php?page=$1 


Security 


FastCGliServer { 
Pas tCGlve — “PHP 
UseToolkit = phprewrite 
# Can be the path to the unix socket too 
ConnectTo eek, ORS UR aie eal Ons 


Extension jOlayo 


Val bisite WHOS: 14 


BOS cies = WWW.example.com 


WebsiteRoot = /usr/local/www/hiawatha 
AccessLogfile = /var/log/hiowatha/example-access.log 


ErrorLogfile = /var/log/hiowatha/example-error.log 


# Althought those directives can provide protection, they are of course 


# not 100% reliable 
[Eigie Vicime\erolnde ves 


Pigs Wrote SONAL ves 


PREVEnNEXS5.— Ves 

# A body which matches this regex is forbidden 

DenyeoCy = 9 2 ooCSCrlLpe.  .5Co7PSeript ooh. > 
Use Tesi ciG ll Nes) 

TimeForCGI 5 


# By default it is index.html 


Security 


StartFile = index.php 


So, for the moment, nothing special and never seen before. Let's dig in more ... Indeed, with Hia- 
watha, we can set some banning policies like below: 

# If in our VirtualHost above a user had the bad body content 

BanOnDeniedBody = 300 

# If too much malformed HTTP requests are made by a client, it’s banned for 1 min. 

BanOnGarbage = 60 

# Banned if the request size exceeds this size 

BanOnMaxRegsize = 512 


# Anti flood measure, here if the client does more than 20 requests per second, it’s banned for 
1 min. 


BadOnFlooding = 20/1:60 

BanOnSQLI = 60 

KickonBan = yes 

# Reset the ban times if the client attempts to connect when it is already banned 
RebanDuringBan = yes 

GarbageLogFile /var/log/hiawatha/garbage.log simple 


Also Hiawatha provides a separated feature, called Monitor, a feature a bit “a la” Munin but well 
adapted to this server. 


# This IP address will be allowed to download the event log files for its analytics 


Monitorserver =2197. 6s 1. 2 


As mentioned before, Hiawatha does not support third party modules, so if you wish for your serv- 
ice to make a decision based on country per IP with geoip, for example, you can use Hiawatha as 
a Reverse Proxy in front of Nginx. It’s as simply as it is shown below inside your VirtualHost set- 
ting. 


NWemiaeibiclliskessice | 
Hostname = Www.myreverseproxy.com 
PreveneC one syes 
Pmevenie.> © lal ves 
PRevVeme Goo. ses 


ReverseProxy .* http://192.168.1.2:8111/geoip 


# Cache internally those contents from reverse proxy requests per extension 


CacheRProxyExtensions = gif,png,j]peg,css 


CacheSize = 256 


3. Conclusion 


Hopefully, this article will give you the curiosity to consider this approach. For what it provides, it 
is impressive regarding the fact that it is a one man work done since 2002 (by Hugo Leisink). 


BSD 


MAGAZINE 


72 


NodeJS and FreeBSD - Part 1 


Nodejs is well known to allow building server applications in 
full JavaScript. 


In this article, we’ll see how to build nodejs from source 
code on FreeBSD. You will need autoconf tools, GNU make, 
Python, linprocfs enabled and libexecinfo installed. GCC/ 
G++ compiler suite (C++11 compliant, ideally 4.8 series or 
above) or possibly clang can be used to compile the whole 
source. 


To start, we need the nodejs source code from this url hitp:/,www.nodejs.org/dist/atest where we 
can find this archive (during the article writing, the last version known is 0.12.2), 
node-v<version>.tar.gz. 


Be prepared to be patient, you have enough time for a cup of coffee, the compilation time needed 
can be quite long... 


Once downloaded and extracted, the famous command trio needs to be typed: 

« ./configure --dest-os=freebsd 

* gmake 

e gmake install 

It’s pretty straightforward on first glance. On FreeBSD, when v8 is compiled we get some compila- 


tion errors: 


Clang ti) DV oe" TARGET ARCH X64) a DENABLE DLS ASSEMBERR + °—DENAB IE CHAt— 
DLE ZAPPING' -I../deps/v8 -pthread —-Wall -Wextra —Wno-unused- 
parameter -m64 -fno-strict-aliasing -I/usr/local/include -03 


~EtUuniGerOn— sec hlOnsm tOAta ase CE LoOnS so ENo- Omit ii admMe- OG imiGer wo Edd tas 
SSCLiIOnSs --fPEUumelloOn=seCcL lone =o. —=fio=t tia —i1e -excepri onic. -MMpe-MpE 
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NodeJS 


/root/node-v0.12.2/out/Release/.deps//root/node-v0.12.2/out/Release/o 
bj.target/v8 libbase/deps/v8/src/base/platform/platform-freebsd.o.d.r 
avin Cy =o 

/root/node-v0.12.2/out/Release/obj.target/v8 libbase/deps/v8/src/base 
/platform/platform—freebsd.o 

7 COEDS) Voy Src bese platform) plAattorm——recosd. cc 


-./Geps/738/src/base/plattorm/plattorm—freeosd. cer 15o:11* errors mem— 


ber reference base type 'int' is not a structure or union 


Pesu lt elstubaek (shared OraryACdress (Sra e On Dabh, scam, 


Pe) 060s /787 Sec/ base, plat form, plattorm kresosd. ce: ol SS. errors use 
of Vung eclared Agere Le ters MAR NORE ORR VE 


MAP PRIVATE | MAP ANON | MAP NORESERVE, 


aN 


../deps/v8/src/base/platform/platform-freebsd.cc:263:48: error: use 
CEeUnCee hanred Presi hic tay MAP ONG hie ny Ea 


MAP PRIVATE | MAP ANON | MAP NORESERVE, 


AN 


io/ JE@S/ 7S/Sve/base/ platform plattorm—freehsd. ce:20 le 40s werror:suse 
Cmiindechared igen ater y MAP SNOKMS kin VE 


MEP eee |e CANON Mar NORE omy eevee es Pe Cr iy 


aN 


4 errors generated. 


Ok, so a result variable ought to be a std::vector but it’s considered wrongly as an int and further- 
more a wrong mmap flag is used. Let’s fix it! 
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std: :vector<SharedLibraryAddress> result; 


SIBCUE de Seroiihone. same, Nee. ING Min alae Marie 


int fd = open("/proc/self/maps", O RDONLY) ; 


hier Che 0) ei eile Ces tile 
while (true) { 
@laisve ysiele na Joneses Syen| LAL 
acelie omit ber (0)) 
clolohia: sib ie —req[ lt] 
enOlS ha Webuser (iON 
ale Gel ers b Mie St aSeleli(amele Relclohe Leva ces a) gar.) 
ii Mcosmihia to) Mp reaky 
UiSLQed stati more lononG add cum ner)y, 
reper OUlger ry wie evelol, wie xcWe elas Jelbnmim wm van Hy yh 
Le eres tes )) break; 
BIE UGelolole Ne eurumsallyal| © fax ie) lenatsietl <r 
1S Gis Ss ctsrete (eel. elelolie ebeisie ae Va, wedi 
(ee eS Mle eee.) mois Kee 
Une lgnedeendt—s shiny PolLong(acdet sour sen )q 
Char bul rer (MAP SlENG Er; 


1 Oy Se See ead ee 


DVpesn ean, 


‘Sene ONie Serge Scie VE le Ir pea) 


NodeJS 


break; 


ig ckoyo le, eg cieleMnel. SeWima cia. ae Ney aricton qcvelele TIL) y 


Apparently, there are two different variables with the same name. Let’s rename the second, the 
int type, to res, for example so the vector result variable can legitimately call push_back method. 
That fixes the first error. 


std: :vector<SharedLibraryAddress> result; 


SHB cuesne: lerolghsim, sag, IVa. Ie INGMBel es lee. ee 


int fd = open("/proc/self/maps", O RDONLY) ; 


tame, “mel S108) Gacueib neigh aa cvenb lacy 
while (true) { 
lace ccholole: doshas ie) AbAL i 
addry purse he ri] 
elolole: peibhmie re) Au) 
addriounren bhi i — a0 
Ge eoisiet giershol(MimelM Telelela Vel umincna® re Pare is.) 
if (res < 8) break; 
Uist Inedectatt a ott ii lobond (adur visit Ber). 
(ashen = idles len ieele- yele lola sow ummsna. ae Var kh 
if (res < 1) break; 
le Westoloha enemies crali ad) oe OAT NE Yona syel) cr 
peach EG waded oU Liem ih 7 ieG.) 


(ress. ojo reak= 


NodeJS 


Dholenbe nelevol Aeagcl, =) (Siena kiohonlbonkyeialeMcetoloba Veibhaid=ye)) + 
Chats biter MAP hee Er iy 


INGE: TOR Wicker. ai Svaiel Stem 17, 


BYEesoreads |, 
ibe, MUCnclerst be cyelsl STUN ee Mi aeMisk t= shy) 
break; 


ie chey fe tel —roho mes Gelb hash —ie Mare No amo iseireleh a Alyy: 


Let’s have a look at the mmap problem. 


MAP NORESERVE is a specific flag which guarantees no swap space will be used for the map- 
ping. However, it is a flag usable on Linux and Solaris /SunoOs. 
mmap (OS: :GetRandomMmapAddr (), 
size, 
PROT NONE, 
MAP PRIVATE | MAP ANON | MAP NORESERVE, 
kMmapFd, 
kMmapFdOffset) ; 
— 
mmap (OS: :GetRandomMmapAddr (), 
SaZey 
PROT NONE, 


MAP PRIVATE | MAP ANON, 


NodeJS 


kMmapFd, 
kMmapFdOffset) ; 
void* reservation = mmap(0OS::GetRandomMmapAddr (), 
LeEGUCS taal ae, 
PROT NONE, 
MAP PRIVATE | MAP ANON | MAP NORESERVE, 
kMmapFd, 


kMmapFdOffset) ; 


void* reservation = mmap(OS::GetRandomMmapAddr(), 
Regus sie size, 
PROT NONE, 
MAP PRIVATE | MAP ANON, 
kMmapFd, 


kMmapFdOffset) ; 


Once modified in every mmap call, we can now retry compiling. However, we get another compila- 
tion error. This time, it casts a pthread self returns call to an int. 


deps/vo/stc/ base, plat torm/plattorvm—-oocis cess sla iie neruon: 
SHeroNS der MeeVee. Phuaeiihe GOnelgtia(tels Ie) « Aecd ack Vyohelenalstsvol Few )) piwles. Marbig Me alc) Gatley Veulie 
lowed 


TEs ONCIGL Toller WedbOk Verhore ali@hes solace le. esvoulhim (()/))e 


The problem is, on FreeBSD, a pthread_t type is not an integral type at all but an opaque struct. 


Instead, we might replace this line by: 


IGS NaIae Aeherscmlien eros ec iMiaie al imchliehecnaoneoym forks rece idl siege nese Selig evelal ser oabne Ate 


Now we are finally able to compile. After a couple of minutes, it is finished but we have still one 
source to update: 1ib/dns.js. Add these two lines after line 127: 


if (process.platform === 'freebsd' && family !== 6) 


hints &= ~exports.V4MAPPED; 


Because FreeBSD does not support this flag, it ought to be cleared. 


This is all for compilation and it is ready to be used. Next time, we'll have an overlook in the appli- 
cation’s building part and ought to see the potential of this library. 
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NodeJS and FreeBSD - Part 2 


Previously, we've seen how to build NodeJS from the 
sources in FreeBSD with minor source code changes. This 
time, we'll have an overview of the application’s build proc- 
ess. 


Numerous excellent tutorials exist to build a nodejs' applica- 
tion in pure Javascript. However, the possibility also exists 
to build an application natively in C/C++. It is exactly what 
we're going to see. 


1. NodeJs application structure 


We only focus on the modern way to build a native application. Before, we had to do a node-waf 
package via a Python script. It is deprecated and was replaced by node-gyp. This is a basic gyp 
project structure: 


A binding.gyp file describes the source code to compile, the package name, eventually the neces- 
sary compilation/linker flags. Let's start with the usual Hello world example, quite FreeBSD. 


2. Hello world 


First, we need an entry point, an initializer, from which we will export our functions to nodejs. 
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NodeJS 


And to register our module. 


NODE MODULE (freebsdmod, Init) => Note that there is no need of a comma after this 
macro 


Very well, but for the moment our module is not useful yet, we would need at least one feature. 


Let's imagine a simple random function that uses, internally, one of our arc4random family func- 
tions, a function that will be called from a nodejs script. The signature of this function would be: 


wedol MevcharsloMiMOelatsne, avaeryOnlGyeeamonglercLiberyel <lhiamme.arncus ayo daptee. 25) i 


We can imagine that, from the nodejs script, we would like to provide a max value limit as a 
unique argument: 
#include <stdlib.h> 


tinclude <node n> => includes. both node and v8 structures 


using namespace v8; 


vVOoldURandom (const) hunct1on@allbackinito<Valic= teargs) 


Isolate *isolate = Isolate::GetCurrent(); => Here, we get the current v8 engine in- 
stance 


unsigned long value = 0; 
Li e(aros ~hengen () eis) 
isolate->ThrowException (Exception: :TypeError ( 
String: :NewFromUtf8 (isolate, "Needs an argument"))); 


if (args[0]->IsNumber()) => the arguments are conveniently wrapped, we 
have access to the caller arguments. 


NodeJS 


Velie w=-Sote terest unsigned sone. (argc 4 random Wier onm (args 0) 
>NumberValue()); 


else 
isolate->ThrowException (Exception: :TypeError ( 


String: :NewFromUtf£8 (isolate, "The argument is not a 
number"))); 


args.GetReturnValue().Set (Number: :New(isolate, value) ); 
} 
VoOddwlinie tance nObjech exports) 


{ 


NODE oil MnO Vexvorrs.8) bandon, Wnandom) ; a= Ne bine inne exporter 
our Random function here 


Now, let's have a look a the binding. gyp file. 


"targets": [ 
{ 


Mire inele (cies Aplsiie Ne Was igciNersteiilelohys en Unciona fei oieleis: Meldks? Agkeliihe “one Genblin pitlelo le 


TSOuUrCeS. tal Ereeosomodece. | 


NodeJS 


Simply, as it is, it is sufficient for this first example. Now, we can compile our module. 


> node-gyp configure 


PenOde=C yorol ard 


We can now test with a simple nodejs script. 


var fmod require ('./build/Release/freebsdmod') ; 
Matrices —VEmoc, ramcomi (tl Ciao ee ll A) eves 


console.log (rnd); => Should print a Significant numerical value 


3. Wrapped objects 


Apart from making atomic C++ functions to export, we also have the possibility to handle more 
complex cases, by making wrapped node objects. For this example, let's use yara library, the mal- 
ware's tool. The binding.gyp file would look like this: 


scepewaxsfeqeree od” Ml 
{ 
Shercliaefone: W@relicnwte aaigewerey glelel=A rr 
MoOUrCeesS SI vyaeranode. ce. |r, 
imo eciree Ww Wests) oc anky mic ice ic 


Wilsenecteniketo re aby jbtote/sileyoriia, Ibibewes) i scliers || 


NodeJS 


A wrapped object must inherit ObjectWrap class. 


#ifndef YARANODE H 


#define YARANODE H 
#include <yara.h> 


ane divlelas Kaghoreksen ier 


#include <node object wrap.h> 


Sie seshe: Bicoahel petclchabukevelouiahenee selohetcie Selstsae “ar hehe, “elenicim veleicne wa Aiiealeloun i 


class YaraNode : public node::ObjectWrap { 
private: 

VReECOMPIERER = *yG; 

iivicny Cealkosy 

explicit YaraNode(); 


~YaraNode(); 


Statrcyvord New (const. Vo shunner tonCallbackinto<v7ss.:Valie>s); 


static v8::Persistent<v8::Function> constructor; => Contrary to the Lo- 
cal handles, a Persistent storage is independant to any HandleScope, valid until cleared 


Statue wold AddRude(consts vos 7Huncitoncelilbackintro<vs ~- Value >.) ; 


NodeJS 


Stables Colin ocanllh ile (COnske wore hiner onea Wilback iit Game teva Wie = wie 
jobheplinkie:+ 
See mVOls Hi (ots hanGle. we Obyecie). 


SHEvsuemie® Cale oy Agee cues ts) h 


The Persistent storage will serve us for the YaraNode initialization from within the Nodejs entry 
point: 


#include "yaranode.h" 


using namespace v8; 


NAOMIS Kole lanbMMelelentmame Keucixeney (Glolausie, Kelaksiay “way camebe, line arey 
Const Chary message, #VOnd  “porivatle) ar, 
Isolate *isolate = Isolate::GetCurrent (); 


if (message) 


isolate->ThrowException (Exception: :TypeError (String: :NewFromUt 


isolate, message))); 


} 


PEerstetent Funct Lon Laranode: const ructon: 


YaraNode::YaraNode() { 


WagsWecue Ns: Gae al gulqamivellial a=.) ur 
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aigie. Gaasiee Meets) —— a Olac TS) UICIG ISIS) © a) 
Ve Tevomonllk Sie Tene Srouuien *Savhe) 


Ve Seto lmsubikong crower ereli cele) <i gers, velelelic al ieleys INIUNL IL) Cr 


YaraNode::~YaraNode() { 
if (yrstatus == ERROR SUCCESS) { 
ViiEeconpa lerndes troy (cc hy 


Vie Sei ere (ye: 


void YaraNode::New(const FunctionCallbackInfo<Value> &args) { 
Isolate *isolate; 
ielers Wintec none wie none 


isolate = Isolate::GetCurrent(); 


HandleScope scope (isolate) ; =>AHandleScope is responsible for all following 
local handles allocations 


if (args.IsConstructCall()) { => var yr =new YaraNode(); 
YaraNode *ynode = new YaraNode(); 


tA iota. estate) SiO su eeitoe) 


NodeJS 


isolate->ThrowException (Exception: :TypeError ( 


String: :NewFromUtf£8 (isolate, "yara could not be in- 
stantiated"))); 


ynode->Wrap (args.This()); => Here we wrap our YaraNode and can be unwrap as 
will as we Il see slighty later 


args.GetReturnValue().Set(args.This()); => We return basically the 
wrapped yaranode object to the javascript caller 


} else { => YaraNode called as classic function 


ClLore—lhoca lohimetion:: New usolace wm constnuceon)-).——- VVve Use 


here our persistent storage to instantiate our YaraNode instance 


args.GetReturnValue() .Set (ctor->NewInstance()); 


void YaraNode: :AddRule(const FunctionCallbackInfo<Value> é&args) { 
Isolate *isolate; 


Ivey Oe sil) y 


isolate = Isolate::GetCurrent(); 
HandleScope scope (isolate) ; 


YaraNode *ynode = ObjectWrap: :Unwrap<YaraNode>(args.Holder()); => 
Here we unwrap to access a YaraNode object field 


Ae eS nT isin, (ilap eo mallu 


AOS Bye Sey 


NodeJS 


for (i = 0; i < args.Length(); i ++) { => addRule method, from nodejs 
script, is called like this addRule(<rule1>.,...,<ruleN>); 


tim, (i Cshareice Wall poe alkcwsnewasligvog ah =f 
CONS bs Glia a tale: 
Serre sUCEs Vabiiewireu © (ates | I= hoo uer bing. (ae; 
rule = *rrstr; 
ia? ee, Velo sijemhlk Sua euellel ave imahigie p(y cabeleker = ery Gable, 4104), 
if (r == 0) 
ynode->yrrules ++; 


Wace’ += YL; 


args.GetReturnValue().Set(Number::New(isolate, yrc)); 


void YaraNode::ScanFile(const FunctionCallbackInfo<Value>& args) { 
Isolate *isolate; 


initia yrecan — 0; 


isolate = Isolate::GetCurrent (); 


HandleScope scope (isolate) ; 


NodeJS 


YaraNode *ynode = ObjectWrap: :Unwrap<YaraNode>(args.Holder()); 


Meew(args. Lengun()w—-— Ie strargs | 0] —-lsSerimng())* eI 


NARS UGS eee 20) 


Cons Chart mle peti, 
LES ynede--yrruless> Utes 


Nae Vevouneni Nie tej o1e se ioilevsn (WanvelelS=e len (ig GNSS) MOIS (Sie 


SHEieal) Cle paUseumieWecblibles watsyenay Geuas te 0) are dhorsmesaalvale pt) ja: 
filepathig—= iri: 
MSN SCNGE as Wie LSS poretsh ain a EARS ie tikots, MeMAMereuelel: ly 


NMOhE ep, INO. Le) i 


} 
ergs. SeERSeuUrn Value ocr. Numbe ms: New (teouaber soy sica i) s)\r 
} 
void YaraNode::Init (Handle<Object> exports) { 
Local<FunctionTemplate> temp; 


Isolate *isolate; 


isolate = Isolate: :GetCurrent(); 
temp = FunctionTemplate::New(isolate, New); 


temp->SetClassName (String: :NewFromUtf8 (isolate, "YaraNode")); => 
From within a nodejs script, the class will have this name, we could have named it differently if 
necessary 


NodeJS 


NODE Zot PoP ROLOT Prev THO (temp,  addnhule! ava raNede = AgCdRimlS)r, => AS 
the single functions with NODE SET METHOD, we expose our methods via this macro 


INOIDIS NSN ib Nes O MOM geds Jib slOlD eee “acstersialsow A leche \elelow wisiere) quello) 


constructor.Reset (isolate, temp->GetFunction()); => Weclear the Per- 
sistent storage for each YaraNode instantation 


exports->Set (String: :NewFromUtf8 (isolate, "YaraNode"), 


temp->GetFunction()); 


void YaraInit (Handle<Object> exports) { 


YaraNode::Init (exports) ; 


NODE MODULE (yara; 4 Yara) 


temp->InstanceTemplate()->SetinternalFieldCount (2); 


We could test this module via this simple nodejs script. 


require('./build/Release/yaranode') ; 


new sm.YaraNode(); 


Ve cocdem ene ai Vemie sek), 


Varese by scomuunl ei a iiblicn Oat a .)us 


We erchercist, (@=p4y, 4 


console.log(ex) ; 


This is a simple example and can of course be greatly improved but that might give you some 
ideas about the possibilities. On several known repositories, a significant amount of native nodejs 
projects already exist that use some popular components (like node geoip, for example). | hope 
this article is able to motivate you enough to start building your own nodejs modules. 
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Cloud service from a developer point of 
view 


In this article, we will have an overview of writing a cloud 
service. Various ways exist to achieve your goals but we will 
focus on one that is memory efficient, multiplatform (POSIX 
systems), multi language (from C++ to Erlang), and reasona- 
bly fast. It is Apache Thrift. | recently fully wrote a cloud serv- 
ice and it worked reliably. 


To illustrate this, we will make a basic remote file handler, 
the server is written in C++ and the client written in Python 
as an example. 


1. Describing the service 


Our server will be able to deliver three different services, listing files or directories, deleting or 
moving a file. Thrift is an IDL (Interface Definition Language) based framework, hence you de- 
scribe your service via an abstract generic language and the Thrift compiler will generate the nec- 
essary code per programming language. The basic Thrift types are all we find in common in all 
languages, byte, binary, integer (i16/32/64), double, boolean, string, some containers as hash- 
map, sets or lists. For those familiar with C and or C++ we can define an atomic file with a 
“struct”: 


The number means the index of the name's field. A file in a UNIX system can have several types, 
not necessarily a regular file but a device, a socket and so forth. 
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So, let's enumerate each type we might need to identify the files, again “a la” C/C++: 


What if we store some file attributes like the size, the permissions bits ... ? Thrift allows you to set 
a struct inside a struct without problems as you can see below: 
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Now, we can start to describe the three Thrift “services” as below; for the first, we would like to re- 
turn a map of files and for the sake of shortening, we “typedef” it as below: 


In addition, for our services we would like to throw an exception in case something went wrong. A 
Thrift exception is very similar to a struct: 


If we do not write the required keyword, a field is then optional. If you re not sure for future devel- 
opment that a field ought to be required, I'd suggest to leave it optional as the clients would stop 
working if the previously required field was suddenly optional in the server's side. 
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Above all of that we might need to customize the language namespace to organize and avoid con- 
flicts; for Java and C++ developers, for example, it is pretty well known. The namespace will be 
translated as well in the target language's logic: 


The first will produce the usual C++'s namespace as: 


Whereas the latter will make the eforensics/cloud Python module. 


In the end, the thrift file might look like this: 
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eee Aa ublye! 
LS aa G. 
i116 mask 
164 size 


SHENG INONCE  SHeNeiileles|< 


SEDC beh bear y 


Pile geyeomn oe 


nea fee Weue sei Gallelcete cou eit 14 


string name 


typedelemap-—s trina bie 7 iile Tilacs & 


Sxeoerloumt mle pexeer rom 


is ule code 


Zee aCuemc o 


Se Glee ead ews e i mcen, 


alge’ plkoltsyc eames’ abers) sill ecto baligiciel is wicehole, \sicne lal) 
EGbeme seep eLonne.), 


throws 


(alms 


2. Generating the code 


Once the service is defined, we can now use the thrift compiler like this: 


In the C++ version, we realize that a skeleton server was generated as well. We will use it to im- 
plement our service! 
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namespace :: cae dade 
namespace :: Rar Giahiaalennalser ang oNaeneereler ie 
namespace :: Re igliagilgi|s Went isroNabeNe lone aa 


Namespaces). : SENT Ss .server-: 


DOGS ee eherecupr, 


namespace ::eforensics::cloud; 


Claes ard hers et vance anichhe tar eee Uap iio lier hems iva eon wy 


joule dkiwes. 


aed S sie igaysal(ete ekeonaie lena) 7 


// Your initialization goes here 


VOlGgFerOLrens Nese ls (it her li sre rete CONC em ou. sri iG soci) 


// Your implementation goes here 


Pranen (Merorensiese s\n. ); 
} 
MVE HOVE eeroOrei=s vec Pri CONS ims ECs tid MCG aie) ary 
// Your implementation goes here 


Pig tie he kotmenc Leo micn il ay 


iive lO gener orehle dCi CONS buss tC. evole GNC Aw oie ec Olle ues Celts 1 iC) s, 
dst) 04 


// Your implementation goes here 


Planer (ono rons les emmys) 


gh owe (tebe (IMeWe wenae(er aie gleka — pxeliete hic) | 
PL OOrLe =. J090> 
elelsecd gee et bemsewVvrcenanea let Wancle ia (Newer nWhen se Avec and lena). 


SaGeGipirk = tEmOCesscor  BOrocessom(lew, bElLotson @ueehmocess On (item — 
fod Neuen ee 


Silared pte 1 Server) Kanspoce- server) Fanspole (new  loerveroock-— 
SEPore) ys, 


shared ptr il Transport bacuory- transporerbaceory (new TBulrered i rans— 
DOrEPaActory () jc 


Silamedeveb trrOcoce HAcrOLy (UrerocoMaAceOouy (New elias ProEroce lh ac— 
LOLy))a 


TSimpleServer server(processor, serverTransport, transportFactory, 
PreeOcolra Gwar.) e 


server.serve(); 


return OQ; 


Now it is up to us to implement the three services. Let's start with the simplest, removing a file 
with the famous C function unlink. 


To improve it, we could make sure the file is a regular file, otherwise return the exception we set 
earlier in the thrift IDL file. 
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meso t= “pach; 
ee Oem COGS (a)e; 
ae  SVels. sien Gilisien) sy: 


Elneow ais 


SEMUCEMS ears, 
Create stat (s,) peach), 


Node Rey ie—ne. oenmoce, 


if ((m & S IFMT) != S IFREG) { 


string msg = "Only files can be removed"; 
foc e ee COGS (=), 
1 Bae tele pilisten (Gqiislep) i 


ikeOws i 


dhim bh Gbugieabicll <i jeyenelen ices) ons) hy 
SeErInG MsGu—-— COUld snore remove)”; 
Ns += spach; 
msg += 


msg += strerror(errno) ; 


ie iat Teerolen(1)- 
Dee pee Qilistes Guliste)\ 7: 


Ehrowr rh. 


ab (GIO. 1S) oreOnal si qelWerei nina (efsiquene« Hoch t swencalLigleks once Welsig sis) IsnelCl ue sueresli@lensa selene) 


{ 


SEMUCES Stats, 
CLOMLSGStat (5, SEC), 


mode tom» —eo .stsmode; 


mene AGU (Sa) CIR UR) a ee ea 


el Coukimer wiser Cig mest ierce fers, lore. uowasel 
ioe cede (= 15, 

Eee Ser liso. (meg); 

throw £; 


} 


Per ences nen mee tic) epee e wee seiag)e) 
String msa.—— Could nor move, \, 


msg += src; 


(sie fegde se SOR a 


Then the last service, listing files or directories. Previously, we defined several types of files and 
their attributes, hence we'll once again rely on the stat function: 
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fl.name = path; 


if ((m & S IFMT) == § IFBLK || (m & S IFMT) —— § IFCHR || 


& S IFMT) == S$ IFIFO) { 
Pa hem eye eye: Win mney, 
& S$ IFMT) == § IFSOCK) { 
EEVept ye Sb ype rn SOChEE, 
& S IFMT) == § IFLNK) { 
Pile e eS sae oe one 
& S IFMT) == S_IFREG) { 

creel Bieler e Oe ee yee anh, 


} else i & S IFMT) == S IFDIR && (((m & S IFMT) & S_IFLNK) 
!= § IFLNK)) 


fey Deena emer ye bys 4. eR bed, 


iS eee eee 


Eve ei ios ke a0 
if (m & S_TWUSR) 
fl.attr.mask |= 0x400; 
if (m & S TRUSR) 
fl acer mask |=) 022007 
sm (tbe Se SEE AORSIRS) 


flvattr mask |= Ux1l00; 


alan 


(m & S IWGRP) 


Pie er amas % Ox040; 
(m & S IRGRP) 
Pia ter mask 
(m & S IXGRP) 
Pawel tik oitas a OxO 0; 
(m & S IWOTH) 
fie erick 0x004; 
(m & S IROTH) 
ad iehelEneraiilelsi.< 
(ys poe EXORE) 


Eater .mask Oe O Oe: 


Char vstrmask(5i> 
SOG Gr S knias km oOCX Jae ee be Macea)E, 


PIC ake see riaske— noe ince eas ky e 


PeSeye) heat a leer iy; 

ibe ai aL cuDebaas yes 
Dine dt Pee penciE(Patie Rohl) a: 
if (dir == NULL) { 


reburn  — i 


SLruce dirent entry, *resul. — NULL; 


// We could have just used readdir but we might need to run it 


// in multi thread context .. 
Wiel he BY (Cice cele iieueie Gali Cem kiya eGo cite) ei) 
if (result == NULL) 
break; 
ier Ss eh ene ay, ce sneha. -Cmiiciie) mea. uml 
SHEN ON AA Alister Olle e el, Saveniite) a) 
continue; 
SHE iaahighe@ mejel-usialy —y jeremslaie 
ane | Gapelen cian belcumianyrsebvas Gi el] 
Poathet— y's 
Ppath ap esult—7 a name; 


FSeocdrenery (Vreeu en, mCP aehy 7 


Closed 1 rican) ee 


return O; 


We are nearly done, let's compile the code. 


If you execute the final executable, it will listen via the 9090 port and if you generated Python's 
version, for example, it should have generated a sample client: 


107 


MAGAZINE 


BSD 


{ '/tmp': file(type=4, attr=file attribute(gid=0, mask=None, uid=0, 
strmask='0777', size=None), name='/tmp'), 


'/tmp/.ICE-unix': file(type=4, attr=file attribute (gid=0, 
mask=None, uid=0, strmask='0777', size=None), name='/tmp/.ICE-unix'), 


‘/emp/.ICE-unix/1997": file(type=2, attr=file attribute (gid=1000, 
mask=None, uid=1000, strmask='0777"', size=None), 
name) emp), Cr —wrais</ 190) 


"/tmp/.XO0-lock': file(type=0, attr=file attribute (gid=0, mask=None, 
uid=0, strmask='0222', size=None), name='/tmp/.X0-lock'), 


"/tmp/.X11-unix': file(type=4, attr=file attribute (gid-—0, 
mask=None, uid=0, strmask='0777', size=None), name='/tmp/.X1ll-unix'), 


'/tmp/.X11-unix/X0': file(type=2, attr=file attribute (gid=0, 
mask=None, uid=0, strmask='0777"', size=None), 
name='/tmp/.X1l1l-unix/X0"'), 


Hy Mey <5 Vdelon.<teleclialLigetemljeve We at mlGey (is qere 7A wecns sige IS eisai 
bute (gid=1000, mask=None, uid=1000, strmask='0700', size=None), 
name='/tmp/.vbox-dcarlier-ipc'), 


“/ tmp) .vbox-dcarlier—-ip]e/iped" ; file(type—2, ater—tile atcri— 
bute (gid=1000, mask=None, uid=1000, strmask='0700', size=None), 
hame="/tmpy -voex-dearlier—-ipc/iped'), 


'/tmp/ .vbox-dcarlier-ipc/lock': file(type=0, attr=file attri- 
bute (gid=1000, mask=None, uid=1000, strmask='0600', size=None), 
Hamne—" jmp, 7oOx-dcarliet—ipe/ lock’); 


'/tmp/config-err-tu3hNl': file(type=0, attr=file attri- 
bute (gid=1000, mask=None, uid=1000, strmask='0600', size=None), 
Hame—="/ Emp, Conmkig-err-eusiNl"), 


‘/tmp/unity support test.0': file(type=0, attr=file attri- 
bute (gid=1000, mask=None, uid=1000, strmask='0662', size=None), 


name='/tmp/unity support test.0') } 


ss enlge On oh On 
DraneDpOoriw— THEEOC hI ene THEr OCI enn (host, oor Ee, aura.) 


else: 


socket = TSSLSocket.TSSLSocket (host, port, validate=False) if ssl 
else TSocket.TSocket (host, port) 


if framed: 
# In this mode, the message is fully read no flush is required 
[Ss SIOKS Oe ne |e TTransport.TFramedTransport (socket) 
else: 
ican c mtr TTransport.TBufferedTransport (socket) 
Prococoly=_ TBinaryerolocol. 1BinmaryeroLrocol(eranspore) 
ClteVep er lewecel ee ele EpuOroce.) 


trans pore, Open) 


# Pretty straightforward to call each server method as you can see 


i eMC — eee ttn ee mike a. 
if len(args) != 1: 
jQueuligie( Sunouasiaculicts. vlc igeeiuliiasics Jl kelgers.. }) 
Syvs.exit (1) 


Pee Prertae (cl tene  etoOteieicsy ts (acca Lola) 


If we come back to the C++ server's code, the skeleton's generated code uses a TsimpleServer, 
which is perfect to start with but is monothread. I'd suggest the TThreadPoolServer (more effi- 
cient than the TThreadedServer) or the TNonBlockingServer instead and to at least add a signal 
handler to terminate the server properly. The TthreadPoolServer's version might look like this: 
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se pele ECS IMEC IER sieigsesvakelaveneliellicna)) & 


Signal (SIGPIPE, servsighandler) ; 


tee ned 


Silapedmeer DrrOocesccor eUrOocessOr MNewre Oludpocry1ecsodimmr boc. 
eso (Mancdler)). 


Saree eE ect yCePre sport scr vCh ME AnopOLe (lew loch VetoOCk= 
Sed (elenasen yy) 


shared tpec Titans pore eaelOry - VeLanePOreP acl Ory (Neve. lsu tered 
Mean SOOrE menor 7) ts 


ShlaGed Penh lPLOtocolm ack omy  sOrol-ocolbacronry (New Ieaharye Goto — 
colFactory()); 


threadManager = 
ThreadManager: :newSimpleThreadManager (workers) ; 


Shaved Der <POsilireedhaceOry ~eetlreagdbacconynew. Os ixinvead— 


Ra Cuorn))).: 
threadManager->threadFactory (threadFactory) ; 


threadManager->start (); 


Stra 3ChOGs oe Server Mis ssra Clinicas ese Co2cCnO lL; 


HServer — shared  per~loerver- (new TihreadPooblerver (proces— 
SOL, Serverlranspolre, EeranspDOrEerFaAcEory, DrIOhkOCcOolFPactouy, elreadhan-— 
elefeie |) e 


nserver->serve (); 


ica Ch WISE exCep redone. ce) om, 


Sec oe LOGm— 8 Ai werr Or FOCeM ree: mi ore Wihiett G) mmo UC emo: 


3. Conclusion 


Apache Thrift works well indeed in most POSIX systems, I've made the full example server part 
of a Linux machine and tested with FreeBSD and Linux. The client was called on a remote 
FreeBSD machine. 


An alternative version exists remade by Facebook called fbthrift which works fully only on Linux 
but the code generated is superior and this version in general has proved to be more efficient in 
term of memory usage at least. There is also Google Protocol Buffer that performs better than the 
two above but has fewer languages supported (officially). You have to write the client / server 
code on your own, though. So based on your own criteria and restrictions, one of these might fit 
better for your own case. 
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